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1. INTRODUCTION 

The idea of bit interleaved coded modulation (BICM) was first introduced in [1] for the case of a single 
antenna and 8PSK modulation. A bit-wise interleaver at a binary convolutional encoder output was used with a soft 
input Viterbi decoder fed by soft bit metrics. It was shown to give a good coding gain over Rayleigh fading channels 
[1], [2]-[6]. BICM approach was extended to a multi input multi output (MIMO) system setup [7], [8]. It is an 
attractive solution since the modulation and coding schemes are selected separately rather than dependently as in 
trellis coded modulation (TCM) of [9]. Muller-Weinfurtner [10], it has been shown that the BICM is superior to the 
space time trellis coded MIMO system of [11], [12] (in fast fading channels). The capacity aspect of such BICM 
systems was studied in [13]. An adaptive BICM variant was suggested in [14]. BICM system with spatial 
component interleaver was proposed in [15]. Various approaches were suggested for detection in such systems [16]- 
[18]. Meeting the requirements of today’s and future wide range of data services relies on the design and 
implementation of reliable, reduced complexity MIMO receivers to utilize the available system resources efficiently. 

The focus of this paper is the design of a soft detector in a BICM spatial multiplexing MIMO system. 
We propose a simple partial tree search based soft detector. This work is organized as follows. Section 2 
presents the system model. Section 3 details the generation of bits log likelihood ratios (LLR’s). Simulation 
results with discussion and the conclusion are in sections 4 and 5, respectively. 


2. SYSTEM MODEL 
The BICM MIMO system is shown in Figure 1, a block (or packet) of source data bits is fed to a 
standard convolutional encoder with code rate R and then interleaved before being mapped to complex 
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symbols by a quadrature amplitude modulation (QAM) modulator. A demultiplexer then splits the symbols 
stream into m branches to be sent by the m transmitter antennas. The n received signals are first processed by 
a soft output MIMO detector that generates soft bits values or LLR’s that are combined together into one 
stream by the multiplexer. The original order is retrieved by a deinterleaver that feeds soft input Viterbi 
decoder. The complex baseband, flat fading equation at any time instant is [19]: 


r=Hs+n (1) 


with r as the n X 1 received vector, s as the m X 1 transmitted vector, H as n X m MIMO channel matrix 
with elements H; ; as the complex channel gains between receiver antenna i and the transmitter antenna j that 
are assumed to be independently and identically distributed (iid) with real and imaginary components as 
Gaussian random variables with zero mean and variance of 0.5. 


Sink 


Figure 1. Bit interleaved coded modulation MIMO system setup 


The n X 1 complex noise vector n is also assumed to have iid elements of zero mean and variance 
a? and a covariance matrix of E(nn’) = o71,, where E is the expectation operator, ( )’ is the conjugated 
transposition of ( ) and I, is the n xn identity matrix. Elements of the vector s are assumed to be 
independently drawn from a gray mapped constellation and scaled to have a unit average power, 
i.e., E(ss’) = Im. The constellation size is æ and the possible transmitted vector combinations are Q = a™. 


3. METHOD 
The sphere detector (SD) [20] is an efficient, depth first, tree search based method to find the 
transmitted vector s that satisfies the condition of: 


min||r — Hs||? 
sen. 


it examines a limited search space of all the Hs points lying inside a sphere of radius r centered at the 
received vector r. Using the structure of the upper triangular matrix R of the QR decomposition of the 
channel matrix H [16]: 


; 2 
mali- Eiti Rigs] a'r (2) 

r =Q'r, VQ = QQ’ = Im and the multiplication by the unitary matrix Q’ does not change the norm 
value of the distance ||r — Hs||?. The algorithm begins at the top level, m of the tree in Figure 2, with the 
selection of the candidate symbol, Sm, that should satisfy the condition: 
<r? 


F 
™ — Rm,mSm 


once such a symbol is found, Sm will be set to it and the search moves down to level m-1, where the condition 
for the selection of the symbol Sm-1, is: 


2 
m 
= RinmSm| + (74 R <r? 
Tm — KmmSm Tm-1 7 m-1jSj| SY 
j=m-1 
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Again, with the first symbol value found to satisfy that condition, s,,_, will be set accordingly and search 
continues downward until the bottom or leaves level of the tree is reached with the constraint: 


m 
ri — > Ry xSx 
k=1 


it is seen from the above that search radius constraint becomes tighter as it scans the tree in downward 
direction since all the terms are larger than zero. The above search can be expressed in term of the branch and 
path metrics; at the top level m, they are the same: 


m 2 


2 
m 
3 ri = Rijs + 
i =i 


i=2 j= 


< r? 


at any other (lower) level, i: 
P; = Pisa + Bi 


the path metric, P;, at the i™ level is the accumulated path metric starting from the top level until the level i 
(included)and it equals the path metric of the parent branch, P;,, plus the branch metric at the level i: 


2 


m 
B; = r- Y Ris; ,L=mm-—1,..,1 


ji 


m 
Pita = yy B; 


j=i+1 


Along any path in the search tree, the resulting metrics are interrelated; a symbol selection at any 
branch (node) will affect all the subsequent metrics. If the path metric at any level i, does not satisfy the 
radius constraint, that branch will be pruned from the tree and not extended further and another symbol 
selection for that branch will be tested for the same parent node or branch; once found it will be extended 
further downward. Once a complete path is found from top to a certain leaf at the bottom, the search radius 
will be reduced to the total path metric of that leaf and search continues with that reduced radius which 
greatly reduces the complexity expressed as the average amount of scanned nodes. At any level, when all the 
possible symbol values (or branches) have been tested for complying with the radius constraint at that level, 
possibly with some pruned, the Algorithm moves up by one level, shifts right by one position and search is 
continued. Eventually, the vector with the smallest path length is nominated as the solution. If no such total 
path is found, the radius is considered to be small; it would be increased and search is restarted. 

It is clear that the complexity is critically dependent on radius selection. It is usually selected based 
on noise properties; since ||r — Hs||? = ||n||?, and ||n||? is a chi square random variable with 2n degrees of 


. . 1 ee : 
freedom, each term of which has a variance of 59°, the radius is set in [21] as r? =kno*,k > 1 where 


n g? as the average value of that variable. Using the inverse cumulative distribution function (CDF) of such 
variable, the radius is selected in [19] as the value that this variable will have (i.e., be smaller than or equal 
to) with a probability that is to be set high enough. There are two important features in the more efficient 
implementation of the sphere detector of [22]. Firstly, the entire possible symbol values are checked at each 
level with the resulting branch lengths (or the partial path metrics) sorted in an ascending order. As such, 
once a path is found that does not satisfy the radius constraint, it will be excluded from the search tree along 
with all its subsequent branches at that hierarchy level. Secondly, the algorithm starts with an infinite radius 
and whenever the point Hs is found to have a path length that is smaller than the radius (including the first 
point), the radius value will be reduced to that metric and search is continued. 
The detector in the non iterative system setup of Figure 1 generates bits LLR’s based on [23]: 


LLR(c;,|r) = 4 min || r — Hs ||? — min || r-H s ||? (3) 
o s:cj=0 s:cj=1 


i= 1,2,...,mM, 
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Level: m, Sm, 
Pm = Bn 


Level: m-1, Sm-1, 
Pma = BnitPm 


Level: 1, sı, 
Pı = Bi+P2 


Figure 2. Sphere detector, dashed lines for branches that do not satisfy radius constraint, thin lines for 
branches yet to be tested, bold lines with arrows indicate one valid complete path 


The minimizations in (3) have to be done (per bit) over all the 2c"? possible bit combinations 
(with the corresponding symbol vector s) that have the bit c; as 0 or 1. Doing an exhaustive search over such 
vectors would be highly inefficient and complex. A list sphere detector (LSD) is proposed where a list is 
constructed that contains the vectors with the least total path metric || r — H s ||? out of an SD that scans the 
full tree [21]. The list vectors are then used in (3). The list size grows with the number of bits per channel 
use, mM, with more reliable LLR value obtained with larger list sizes. Since the LSD has to scan the full 
tree, it is still complex especially for systems with large dimension of Q. Here, we modify the SD of [22] to 
find a limited number of vectors to be used in (3). In this modification, the search radius is selected according 
to n and the noise variance as in [21]. Since the original algorithm is aimed at finding hard output solution 
with the minimum of total path metric || r — H s ||*, it considers only the first branch at the bottom level since 
that would be, with sorting, the one with the least metric. We use all the valid branches at that level to 
construct the list out of a partial tree search and use the first candidate vectors found to satisfy the radius 
constraint. Radius reduction will be disabled and if the number of found vectors at a certain radius is less than 
the list size, then the radius is increased just to look for extra ones to fill the list. All the path metrics for the 
list vectors are fed back to be used for the calculation of the soft values. 


4. SIMULATION RESULTS AND DISCUSSION 

We used the 64 state, rate R = Z, non systematic convolutional code with octal generator polynomial 
of (133,171)s, six bits were used for trellis termination and the decoder trace back length was 35 bits. Block 
fading model is used where the channel stays unaltered for a number of channel uses and then changes, 
randomly and independently, from block to another. Figures 3 and 4 show the complexity in terms of the 
average number of visited nodes with list size L = 64 and 128, respectively for a 4x4 system with 16 QAM. 


4 x 4 MIMO System, 16 QAM, L = 64 4 x 4 MIMO System, 16 QAM, L = 128 
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Figure 3. Average number of scanned nodes Figure 4. Average number of scanned nodes 
4 by 4 system with 16 QAM and L=64 4 by 4 system with 16 QAM and L=128 
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Figures 5 and 6 show the corresponding bit error rate (BER) graphs where the mmse and zero 
forcing (ZF) based detectors refer to those of [23], [24], respectively. Compared with reference LSD method 
of [21], it is seen that our current suggestion reduces the complexity much further with effectively a very 
close performance. For that purpose, an extra cost of sorting is required which is in the order of O(a log a) of 
the (quick sort) algorithm of MATLAB [25]. The net effect is a noticeably faster approach. 

Figures 7 and 8 present a similar comparison in terms of the BER and complexity for the case of a 
2 by 2 system with 64 QAM and L = 128. It is to be noted that search tree for this case is considerably 
smaller than that of Figures 3 and 4; there is 647 = 4096 and 16t = 65536 possible transmitted vector 
combinations for the first and second cases, respectively. The performances of detectors of [23], [24] have 
come closer together as shown in Figure 7 (in comparison to that of Figure 5). Figure 5 shows a gap of about 
1 dB at BER = 107? between the tree search based detectors and that of [23]. 


4 x 4 MIMO System, 16 QAM, L = 64 4 x 4 MIMO System, 16 QAM, L = 128 
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Figure 5. BER vs. SNR for a 4 by 4 system with 16 
QAM and L=64 
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Figure 7. BER vs. SNR for a 2 by 2 system with 64 
QAM and L=128 


5. CONCLUSION 


Figure 6. 
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Figure 8. Average number of scanned nodes 
2 by 2 System with 64 QAM and L=128 


BER vs. SNR for a 4 by 4 system with 16 


An efficient, reduced complexity, partial tree search based soft detector has been suggested for 
BICM MIMO systems. It is shown by simulations that a very close performance is achieved at a much lower 
computational effort in terms of the average number of scanned nodes. It is thought that the suggested 
method is promising to be further utilized in an iterative system setup to be dealt with in a future work. 
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